METHOD AND APPARATUS FOR VIDEO SURVEILLANCE WITH DEFINED ZONES 

CROSS REFERENCE TO RELATED APPLICATIONS 
This application claims the benefit of United States provisional patent application 
5 serial number 60/188,171 filed on March 10, 2000. United States Patent applications, also 
claiming the benefit of U.S. Provisional application no. 60/188,171, and entitled "Method and 
Apparatus for Object Tracking and Detection" and "Method and Apparatus for Object 
Surveillance with a Movable Camera" were filed concurrently herewith. 

10 FIELD OF THE INVENTION 

The present invention relates to the field of video surveillance systems using motion 
video cameras. 

BACKGROUND OF THE INVENTION 
15 There are several shortcomings in current video surveillance systems that need to be 

overcome for widespread use of automatic detection and collection of relevant video data in 
response to scene stimulus without the need of a human operator present. When viewing a 
scene from a video camera a large amount of data is generated. The vast amount of data 
created produces a data reduction problem. Automatically detecting and accurately and 
20 reliably collecting image information of a moving object using a motion video camera is a 
difficult task. This task is made even more difficult when trying to detect, track and maintain 
camera line-of-sight using a single motion video camera without requiring human 
intervention. 

U.S. Patent 5,473,369 (Abe) describes the use of a camera to detect and track a 
25 moving object without using conventional block matching. In the system described in Abe 
single object tracking is performed only after an object is placed within a frame on a screen; 
however, there is no user input device for manual target selection Moreover, Abe does not 
provide for camera movement to maintain line-of-site. 

Other prior art solutions provide for image stabilization for a camera in arbitrary 
30 motion without object tracking functionality. U.S. Patent 5,629,988 (Burt) teaches electronic 
stabilization of a sequence of images with respect to one another but provides no tracking 
facility. 
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Still other prior art solutions control camera movement to maintain line-of-sight 
between camera and object but lack arbitrary motion compensation or do not provide for 
automatic and user selected object tracking. U.S. Patent 5,434,621 (Yu) teaches a method 
for automatic zooming and automatic tracking of an object using a zoom lens but does not 
5 provide for reorienting the camera's line-of-sight. 

SUMMARY OF THE INVENTION 
It is an object of the present invention to provide a video surveillance method and 
system having increased accuracy. 
10 It is an object of the present invention to provide a method and system for reducing 

false alarms in motion video object detection and surveillance. 

It is an object of the present invention to provide a method and system for identifying 
user-defined zones to control motion video tracking. 

In accordance with one aspect of the present invention there is provided a method for 
15 defining a control zone in a field of view of a motion video camera, said method comprising 
the steps of: displaying motion video data representative of the field of view of the motion 
video camera; receiving indication of a control zone type; and receiving indication of a 
control zone size within the field of view of the motion video camera. 

In accordance with another aspect of the present invention there is provided a system 
20 for defining control zones of different types in a field of view of a motion video camera, said 
system comprising: a database containing a description for each of a plurality of control zone 
types; means for defining a control zone in a selected area of the field of view of the motion 
video camera, said control zone being of a type selected from one of said plurality of control 
zone types in said database; and means for displaying a received motion video signal from 
25 the motion video camera including an indication of said defined control zone. 

In accordance with a further aspect of the present invention there is provided a 
computer readable medium having stored thereon computer-executable instructions for 
defining a control zone in a field of view of a motion video camera performing the steps 
comprising: displaying motion video data representative of the field of view of the motion 
30 video camera; receiving indication of a control zone type; and receiving indication of a 
control zone size within the field of view of the motion video camera. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is an exemplary configuration of a video surveillance system having user- 
defined zones according to an embodiment of the present invention; 

Figure 2 shows an exemplary field of view for a camera highlighting user-defined 
5 zones according to the present invention; 

Figure 3 is a system diagram of a zone defining processing system the video 
surveillance system of Figure 1 according to an embodiment of the present invention; 

Figure 4 is a flow chart illustrating a method of identifying user-defined zones 
according to the present invention; and 
10 Figure 5 is a flow chart illustrating a general method for video surveillance using the 

user-defined zones identified by the method in Figure 4. 



DETAILED DESCRIPTION 
Object tracks, derived from motion video data, are a very useful way to automatically 

15 detect alarm conditions within a camera's field of view for purposes such as security and 

surveillance. However, in the field of motion video object tracking, environmental noise and 
other such sources of movement can lead to false tracking and tracking of unimportant 
objects such as leaves falling from a tree. 

The present invention uses user-defined regions within a motion video camera's field 

20 of view to ignore the origination of object track data but allow tracking of objects already 
being tracked before they entered that region. Therefore, the user defines in advance those 
regions that may generate false alarms within a motion video camera's field of view. The use 
of user-defined zones in conjunction with object detection and tracking techniques provides a 
video surveillance system that has greater object tracking accuracy than existing systems. 

25 Figure 1 illustrates a video surveillance system 100 according to an embodiment of 

the present invention. A motion video camera 108 has a field of view 1 18. A computer 102 
receives and processes a video signal 112 from the motion video camera 108 and performs 
object tracking and detection to determine if there was movement in the field of view 118. 
The computer 102 contains a zone defining processing system (Figure 3) for defining 

30 surveillance zones in the camera's 108 field of view. 

For a moveable motion video camera 108, the computer 102 generates a control signal 
1 14 for servo controlled pan-tilt-zoom assembly 110. The control signal 1 14 is based on the 
current position of the servo controlled pan-tilt-zoom assembly 110 and information 
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contained in the video signal 1 12. Such movement allows the motion video camera 108 to 
capture an object of interest in greater detail or improve the camera's 108 line of sight with 
the object of interest. The object detection and tracking techniques that are used can be, for 
example, those taught in Applicant's related applications entitled "Method and Apparatus for 

5 Object Tracking and Detection" and "Method and Apparatus for Remote Object Tracking and 
Detection", filed concurrently herewith, both of which are incorporated herein by reference. 

Object detection may be accomplished using any number of methods for image 
segmentation known in the art. For example, motion detection may be performed by frame 
differencing sequential pairs of video frames and applying thresholding techniques thereby 

10 yielding pixels within the processed image that reflect motion of objects with the field of 
view of the camera 108. Additional image processing techniques such as centroid analysis 
may then be applied to remove spurious motion. Kalman filtering may be applied over time 
to further remove random motion and to estimate motion of objects for the purpose of 
anticipating camera 108 repositioning and maintaining tracking when moving objects are 

15 temporarily occluded by stationary ones. Object tracking and detection is discussed in 

greater detail in applicant's co-pending related applications entitled "Method and Apparatus 
for Object Tracking and Detection" and "Method and Apparatus for Object Surveillance with 
a Movable Camera" filed concurrently herewith and incorporated herein by reference. 

For a fixed motion video camera 108 (not shown) not having a servo controlled pan- 

20 tilt-zoom assembly 110, the computer 102 uses the video signal and the current magnification 
of the camera 108 to create a control signal similar to the control signal of the moveable 
camera 1 14. However, the control signal for a fixed camera only uses a zoom function 
already on the camera 108 to capture the object of interest in greater detail. 

The video signal 112 received from video camera 108 is passed out from the 

25 computer 102 as video signal out 106 either directly to a display 104 or modified to include 
graphic information that may be used to set up response parameters of a tracking program, 
indicate an object that is actively being tracked or identify (or allow identification of) user- 
defined zones for tracking. A pointing device 116 may be a mouse or trackball and is the 
user input for modifying said response parameters or defining the tracking zones. The 

30 pointing device 1 1 6 may also be used by the user to select an object that appears within the 
field of view 118 such that tracking program residing on the computer 102 acknowledges 
user's selection and initiates tracking of the selected object. 
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Figure 2 is an illustration of an exemplary field of view 200 for a camera highlighting 
user-defined zones according to the present invention. In this illustration, the environment 
that the motion video camera 108 is receiving within its field of view 200 is shown. For the 
purpose of illustration, the environment contains a house 220, a tree 230, a walkway 240, and 
a pond 250. There are five different types of zones 202, 204, 206, 208, and 210 that control 
tracking behavior and subsequent video output. These zones are defined as follows: 



Zones 


Definition 


Application 


Tracking Zone 


Sets the overall surveillance region 
defining pan and tilt limits oi camera. 
The tracker does not track nor does it 
move the camera beyond the tracking 
zone. 


Include only the region 
in which tracking is 
required. 


Black-out Zone 


Sets regions within the tracking zone in 
which tracker will not track. 1 he 
tracker does not track any target that 
moves into a black-out zone, nor does 
it originate a target within a blackout 
zone. 


Reflections, high traffic 
areas, machinery, other 
unwanted distractions. 


Exclusion Zone 


Sets regions within the tracking zone in 
which tracker will not originate a new 
tracK dul xnai xracKea oojecis may cmci 
and exit. The tracker maintains a track 
that enters an exclusion zone. 


Trees, machinery, water 
and other stationary 
rpflprtivp surfaces 


Entry Zone 


Sets region within the tracking zone in 
which tracker will automatically 
originate a new track and follow. 
When the tracker is programmed to 
search only the entry zone, then this is 
the only region in which tracks can 
originate. 


Watch doors, windows, 
vehicles and other 
assets. 
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Privacy Zones 


Sets regions in the tracking zone not 


Areas within tracker 




viewable by the operator. The tracker 


field of view but 




maintains tracking passing into and out 


requiring privacy from 




of privacy zones. 


camera such as office 






windows and homes. 



The tracking zone 208 is shown on the display encompassing the house 220, the tree 
230, the walkway 240, and the pond 250. Only for the purpose of illustration is the tracking 

5 zone 208 contained within a single field of view 1 1 8. For embodiments using a moveable 
camera 108, the tracking zone 208 may span several fields of view. An exclusion zone 250 is 
shown encompassing that portion of the pond 250 that is within the tracking zone 208. A 
black-out zone 202 is shown encompassing the leafy portion of the tree 230 which may 
generate tracking events when the leaves move in the wind. Since in this field of view 200, it 

10 is not anticipated that tracks will not enter into the perimeter, the entire region receives a 
black-out zone 202 rather than an exclusion zone 204. An entry zone 206 is shown 
encompassing the door of the house 220. A privacy zone 210 is shown encompassing the 
window of the house 220. 

Figure 3 is a zone defining processing system 120 for defining surveillance zones in 

15 the camera's 108 field of view according to an embodiment of the present invention. The 
zone defining processing system 120 has a field of view (FOV) area definer 132 that connects 
interfaces to various devices (i.e. camera 108, input device 116, and display 104), a zone type 
database 128 and a tracking/monitoring controller 130. A camera interface 122 receives the 
video signal in from the motion video camera 108 and passes this signal to the FOV area 

20 definer 132. The FOV area definer 1 32 can pass this signal directly to a display interface 1 26 
to be shown on the display 106 or the signal may be modified to include a graphic overlay 
containing information on zones within the field of view. 

The FOV area definer 1 32 may send a zone type selection menu to the display 
interface 126 to prompt a user to select a zone type using the input device 116. The zone type 

25 database 128 contains definitions for the different types of zones as well as corresponding 
actions for each zone type (i.e. do not initiate tracking objects but continue following 
previous tracks). Indication of a selected zone type is received at a input device interface 
124. 
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After a zone type has been selected the FOV area definer 132 provides the display 
interface 126 with a graphic overlay for the field of view to assist the user in drawing a zone 
of the selected type. After an indication of a drawn zone has been received through the input 
device interface 124, the FOV area definer 132 provides the display interface 126 with a 
5 graphic overlay indicating the defined zone. 

The tracking/monitoring controller 130 tracks and monitors defined objects within the 
field of view. Given a field of view the tracking/monitoring controller 1 30 consults the FOV 
area definer 132 to determine a mapping between defined zones and the field of view. The 
FOV area definer 132 provides the tracking/monitoring controller 130 with the mapping 

10 between defined zones and the field of view as well as a definition of the zone types and the 
corresponding actions for each zone type. 

Figure 4 is a flow chart illustrating a method 300 of identifying user-defined zones 
according to the present invention. All of these zones are shown on the display 104 as 
graphic overlays to the video signal in 1 12 received from the camera 108 in step 302. An 

15 input device such as a mouse 1 16 is provided for selecting a zone type in step 304. The type 
of zone may be selected, for example, from a menu listing all zone types or applicable zone 
types given other settings. These zones may be selected and resized in step 306 by a user 
with a mouse 1 16 or similar pointing device by holding down a mouse button and moving the 
mouse 1 16 to resize a rectangular region over some portion of the display's image. If the 

20 extent of a rectangular region exceeds a single field of view, the mouse 1 16 is moved to any 
of the four edges or corners of the display which results in the tracking program residing on 
the computer 102 sending control signal 1 14 to the pan-tilt-zoom assembly 1 10 resulting in 
an altered field of view. The mouse button is released when the desired region is 
encompassed by a rectangle. After the size of the zone has been drawn in step 306, the 

25 method 300 is repeated for multiple instances of zones of a single type, as applicable, as for 
other types of zones. Multiple overlapping and non overlapping shapes may be used to 
define more complex zones. Once these regions are defined, the tracker performs in its 
environment according to the rules that are defined for each type of region. 

Fig. 5 is a flow chart illustrating a general method 400 for video surveillance using the 

30 user-defined zones identified by the method 300 in Figure 4. The video surveillance system 
continuously monitors a tracking zone (except for any black-out zones) in the field of view 
for movement 402. When movement is detected 404 subsequent video frames are compared 
406 to determine what zone the movement occurred in 408. If the movement is not in a 
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black-out zone or exclusion zone the object is isolated 410 and tracking starts 412. If the 
movement is in a black-out zone or exclusion zone tracking is not initiated and the video 
surveillance system continues to monitor the tracking zone for movement 402. While an 
object is being tracked if it leaves the current field of view and enters a new field of view 414 

5 the camera 108 may be moved 418 to center the tracked object in the camera's 108 field of 
view. Movement of the camera 108 to maintain a moving object in the field of view is 
discussed in more detail in applicant's co-pending application related applications entitled 
"Method and Apparatus for Object Surveillance with a Movable Camera" filed concurrently 
herewith and incorporated herein by reference. If the object enters a black-out zone or exits 

10 the tracking zone 416 then tracking stops and the system continues to monitor the tracking 
zone for movement 402. If the object has not entered a black-out zone or exited the tracking 
zone then the object continues to be tracked. 

If movement is not detected in step 404 then the camera 108 is moved to the next field 
of view 403 to continue searching for movement 402. 

15 It is apparent to one skilled in the art that numerous modifications and departures 

from the specific embodiments described herein may be made without departing from the 
spirit and scope of the invention. 
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